Dynamic programming with state-dependent discounting

Persistently Optimal Policies in Stochastic Dynamic Programming with Generalized Discounting

In this paper we study a Markov decision process with a non-linear discount function. Our approach is in spirit of the von Neumann-Morgenstern concept and is based on the notion of expectation. First, we define a utility on the space of trajectories of the process in the finite and infinite time horizon and then take their expected values. It turns out that the associated optimization problem l...

Countable-State, Continuous-Time Dynamic Programming with Structure

Dynamic Programming Solution of State Estimation Problems with Constrained Disturbances

This paper is concerned with maximum a posteriori state estimation of linear systems in the presence of constrained scalar process noise. The goal of this work is to investigate closed form solutions aimed at reducing the online computations required by the estimation problem. Dynamic programming is used to derive a closed form solution that can be precomputed offline. The optimal solution is g...

Hybrid Methods for Continuous State Dynamic Programming

We propose a method for solving continuous-state and action stochastic dynamic programs that is a hybrid between the continuous space projection methods introduced by Judd and the discrete space methods introduced by Bellman. Our hybrid approach yields a smooth representation of the value function while preserving the computational simplicity of discrete dynamic programming. The method is espec...

State Indexed Policy Search by Dynamic Programming

We consider the reinforcement learning problem of simultaneous trajectory-following and obstacle avoidance by a radio-controlled car. A space-indexed non-stationary controller policy class is chosen that is linear in the features set, where the multiplier of each feature in each controller is learned using the policy search by dynamic programming algorithm. Under the control of the learned poli...